Panoramic video is becoming increasingly popular. With the availability of inexpensive panoramic cameras, it is becoming commonplace for consumers to record both still images and video. These panoramic cameras are often limited by their resolution, frame rate, and consumer-grade optics. They are typically designed to record video to a memory storage device (e.g., SD micro) for playback at a later time. Broadcast quality cameras, whether panoramic or not, are characterized by high quality optics, high frame rates and resolutions, and adherence to broadcast input/output standards, such as SMPTE 292M, as well as the ability to stream the video in real-time. This is requisite for live broadcasts.
Presently, there is a paucity of panoramic broadcast cameras in the market. The few examples in the market rely on a plurality of cameras to capture the panorama. The video from the plurality of cameras is then off-loaded to a computing engine, where the images are “stitched” into a composite panorama. Additional processing is required to account for the differences in black levels, white balance, CMOS sensor performance, etc. Because of this processing burden, high frame rates are difficult to achieve. As broadcasters move to higher frame rates—60 fps is becoming the norm—the video processing becomes untenable, let alone the ability to augment the panorama in real time.
Augmented reality, likewise, has become commonplace. Numerous technologies exist to enhance broadcast video. One example is Sportvision (www.sportvision.com) which “overlays” graphical objects such as the first-down line for American football; this augmentation is generated in real-time and composited with the video. This conventional system uses motion sensors on multiple broadcast cameras as well as an enhanced version of Chroma key technology (often used for green screen or blue screen backgrounds). Thus, the conventional video augmentation requires a plurality of interacting devices, as well as a complicated calibration process. Moreover, the video processing and overlays are performed on a conceptual two-dimensional space.
Similarly, augmented reality is employed in gaming consoles, where objects, often corresponding to advertising, or avatars, are graphically constructed in real time by the gaming engine. These experiences intended to mimic reality are actually pre-programmed “maps” through which the gamer may navigate.
Other conventional augmented reality applications include smart phone “apps” where a photo taken on the device is augmented. An example would be an app that allows the user to “try on” various eyeglasses, using a facial photo acquired with the mobile device. With few exceptions, most augmented reality experiences are manufactured bespoke applications.